<!DOCTYPE html>
<html>
<head>
  <meta charset="utf-8">
  

  
  <title>finding_donors_2nd | Chen_Blog</title>
  <meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1">
  <meta name="description" content="机器学习项目为CharityML寻找捐献者 监督学习项目介绍：在这个项目中，你将使用1994年美国人口普查收集的数据，选用几个监督学习算法以准确地建模被调查者的收入。然后，你将根据初步结果从中选择出最佳的候选算法，并进一步优化该算法以最好地建模这些数据。你的目标是建立一个能够准确地预测被调查者年收入是否超过50000美元的模型。这种类型的任务会出现在那些依赖于捐款而存在的非营利性组织。了解人">
<meta property="og:type" content="article">
<meta property="og:title" content="finding_donors_2nd">
<meta property="og:url" content="http://yoursite.com/2018/07/10/finding-donors-2nd/index.html">
<meta property="og:site_name" content="Chen_Blog">
<meta property="og:description" content="机器学习项目为CharityML寻找捐献者 监督学习项目介绍：在这个项目中，你将使用1994年美国人口普查收集的数据，选用几个监督学习算法以准确地建模被调查者的收入。然后，你将根据初步结果从中选择出最佳的候选算法，并进一步优化该算法以最好地建模这些数据。你的目标是建立一个能够准确地预测被调查者年收入是否超过50000美元的模型。这种类型的任务会出现在那些依赖于捐款而存在的非营利性组织。了解人">
<meta property="og:locale" content="default">
<meta property="og:updated_time" content="2018-07-13T03:08:44.768Z">
<meta name="twitter:card" content="summary">
<meta name="twitter:title" content="finding_donors_2nd">
<meta name="twitter:description" content="机器学习项目为CharityML寻找捐献者 监督学习项目介绍：在这个项目中，你将使用1994年美国人口普查收集的数据，选用几个监督学习算法以准确地建模被调查者的收入。然后，你将根据初步结果从中选择出最佳的候选算法，并进一步优化该算法以最好地建模这些数据。你的目标是建立一个能够准确地预测被调查者年收入是否超过50000美元的模型。这种类型的任务会出现在那些依赖于捐款而存在的非营利性组织。了解人">
  
    <link rel="alternate" href="/atom.xml" title="Chen_Blog" type="application/atom+xml">
  
  
    <link rel="icon" href="/favicon.png">
  
  
    <link href="//fonts.googleapis.com/css?family=Source+Code+Pro" rel="stylesheet" type="text/css">
  
  <link rel="stylesheet" href="/css/style.css">
</head>

<body>
  <div id="container">
    <div id="wrap">
      <header id="header">
  <div id="banner"></div>
  <div id="header-outer" class="outer">
    <div id="header-title" class="inner">
      <h1 id="logo-wrap">
        <a href="/" id="logo">Chen_Blog</a>
      </h1>
      
        <h2 id="subtitle-wrap">
          <a href="/" id="subtitle">业精于勤，荒于嬉；行成于思，毁于随。</a>
        </h2>
      
    </div>
    <div id="header-inner" class="inner">
      <nav id="main-nav">
        <a id="main-nav-toggle" class="nav-icon"></a>
        
          <a class="main-nav-link" href="/">Home</a>
        
          <a class="main-nav-link" href="/archives">Archives</a>
        
      </nav>
      <nav id="sub-nav">
        
          <a id="nav-rss-link" class="nav-icon" href="/atom.xml" title="RSS Feed"></a>
        
        <a id="nav-search-btn" class="nav-icon" title="Search"></a>
      </nav>
      <div id="search-form-wrap">
        <form action="//google.com/search" method="get" accept-charset="UTF-8" class="search-form"><input type="search" name="q" class="search-form-input" placeholder="Search"><button type="submit" class="search-form-submit">&#xF002;</button><input type="hidden" name="sitesearch" value="http://yoursite.com"></form>
      </div>
    </div>
  </div>
</header>
      <div class="outer">
        <section id="main"><article id="post-finding-donors-2nd" class="article article-type-post" itemscope itemprop="blogPost">
  <div class="article-meta">
    <a href="/2018/07/10/finding-donors-2nd/" class="article-date">
  <time datetime="2018-07-10T06:53:42.000Z" itemprop="datePublished">2018-07-10</time>
</a>
    
  </div>
  <div class="article-inner">
    
    
      <header class="article-header">
        
  
    <h1 class="article-title" itemprop="name">
      finding_donors_2nd
    </h1>
  

      </header>
    
    <div class="article-entry" itemprop="articleBody">
      
        <h1 id="机器学习项目"><a href="#机器学习项目" class="headerlink" title="机器学习项目"></a>机器学习项目</h1><p>为CharityML寻找捐献者</p>
<h2 id="监督学习"><a href="#监督学习" class="headerlink" title="监督学习"></a>监督学习</h2><p>项目介绍：<br>在这个项目中，你将使用1994年美国人口普查收集的数据，选用几个监督学习算法以准确地建模被调查者的收入。然后，你将根据初步结果从中选择出最佳的候选算法，并进一步优化该算法以最好地建模这些数据。你的目标是建立一个能够准确地预测被调查者年收入是否超过50000美元的模型。这种类型的任务会出现在那些依赖于捐款而存在的非营利性组织。了解人群的收入情况可以帮助一个非营利性的机构更好地了解他们要多大的捐赠，或是否他们应该接触这些人。虽然我们很难直接从公开的资源中推断出一个人的一般收入阶层，但是我们可以（也正是我们将要做的）从其他的一些公开的可获得的资源中获得一些特征从而推断出该值<br><br>这个项目的数据集来自<a href="https://archive.ics.uci.edu/ml/datasets/Census+Income" target="_blank" rel="noopener">UCI机器学习知识库</a>。这个数据集是由Ron Kohavi和Barry Becker在发表文章”Scaling Up the Accuracy of Naive-Bayes Classifiers: A Decision-Tree Hybrid”之后捐赠的，你可以在Ron Kohavi提供的<a href="https://www.aaai.org/Papers/KDD/1996/KDD96-033.pdf" target="_blank" rel="noopener">在线版本</a>中找到这个文章。我们在这里探索的数据集相比于原有的数据集有一些小小的改变，比如说移除了特征’fnlwgt’ 以及一些遗失的或者是格式不正确的记录。</p>
<h2 id="可视化处理脚步"><a href="#可视化处理脚步" class="headerlink" title="可视化处理脚步"></a>可视化处理脚步</h2><p>see the bottom python script</p>
<h2 id="探索数据"><a href="#探索数据" class="headerlink" title="探索数据"></a>探索数据</h2><p>运行下面的代码单元以载入需要的Python库并导入人口普查数据。注意数据集的最后一列’income’将是我们需要预测的列（表示被调查者的年收入会大于或者是最多50,000美元），人口普查数据中的每一列都将是关于被调查者的特征。<br><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 检查你的Python版本</span></span><br><span class="line"><span class="keyword">from</span> sys <span class="keyword">import</span> version_info</span><br><span class="line"><span class="keyword">if</span> version_info.major != <span class="number">3</span> <span class="keyword">and</span> version_info.minor != <span class="number">5</span>:</span><br><span class="line">    <span class="keyword">raise</span> Exception(<span class="string">'请使用Python 3.5来完成此项目'</span>)</span><br></pre></td></tr></table></figure></p>
<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 为这个项目导入需要的库</span></span><br><span class="line"><span class="keyword">import</span> numpy <span class="keyword">as</span> np</span><br><span class="line"><span class="keyword">import</span> pandas <span class="keyword">as</span> pd</span><br><span class="line"><span class="keyword">from</span> time <span class="keyword">import</span> time</span><br><span class="line"><span class="keyword">from</span> IPython.display <span class="keyword">import</span> display <span class="comment"># 允许为DataFrame使用display()</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 导入附加的可视化代码visuals.py</span></span><br><span class="line"><span class="keyword">import</span> visuals <span class="keyword">as</span> vs</span><br><span class="line"></span><br><span class="line"><span class="comment"># 为notebook提供更加漂亮的可视化</span></span><br><span class="line">%matplotlib inline</span><br><span class="line"></span><br><span class="line"><span class="comment"># 导入人口普查数据</span></span><br><span class="line">data = pd.read_csv(<span class="string">"census.csv"</span>)</span><br></pre></td></tr></table></figure>
<h3 id="数据探索"><a href="#数据探索" class="headerlink" title="数据探索"></a>数据探索</h3><p> 首先我们对数据集进行一个粗略的探索，我们将看看每个类别里会有多少被调查，并且告诉我们大于50000美金一年收入的比例。</p>
<ul>
<li>总的记录数据，<code>n_records</code></li>
<li>年收入大于50，000美金的人数 <code>n_greater_50k</code></li>
<li>年收入小于等于50，000美金人数,<code>n_at_most_50k</code></li>
<li>年收入大于50，000美金人数的占比，<code>greater_percent</code></li>
</ul>
<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 总的记录数</span></span><br><span class="line">n_records = len(data)</span><br><span class="line">n_greater_50k = len(data[data[<span class="string">'income'</span>]==<span class="string">'&gt;50k'</span>])</span><br><span class="line">n_at_most_50k = len(data[data[<span class="string">'income'</span>]==<span class="string">'&lt;=50k'</span>])</span><br><span class="line">greater_percent = n_greater_50k/n_records</span><br><span class="line">print(<span class="string">"n_greater_50k values Rate:"</span>,greater_percent)</span><br></pre></td></tr></table></figure>
<h2 id="准备数据"><a href="#准备数据" class="headerlink" title="准备数据"></a>准备数据</h2><p>在数据能够被作为输入给机器学习算法之前，需要对原始数据进行清洗，格式化和重新组织，通常这些操作被称为预处理。本次数据不存在缺失值，所以没有必要处理无效和丢失值。</p>
<h3 id="获得特征和标签"><a href="#获得特征和标签" class="headerlink" title="获得特征和标签"></a>获得特征和标签</h3><p>income 列是我们需要的标签，记录一个人的年收入是否高于50k。因此需要剔除出来。<br><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># split datasets  features and label</span></span><br><span class="line">income_raw = data[<span class="string">'income'</span>]</span><br><span class="line">features_raw = data.drop(<span class="string">'income'</span>,axis=<span class="number">1</span>)</span><br></pre></td></tr></table></figure></p>
<p>###<br><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">vd.distribution(features_raw)</span><br></pre></td></tr></table></figure></p>
<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">skewed = [<span class="string">'capital-gain'</span>,<span class="string">'capital-loss'</span>]</span><br><span class="line">features_raw[skewed] = data[skewed].apply(<span class="keyword">lambda</span> x: np.log(x+<span class="number">1</span>))</span><br><span class="line">vs.distribution(features_raw,ttransformed = <span class="keyword">True</span>)</span><br></pre></td></tr></table></figure>
<h3 id="Normalized-numberical-value-features"><a href="#Normalized-numberical-value-features" class="headerlink" title="Normalized numberical value features"></a>Normalized numberical value features</h3><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">from</span> sklearn.preprocessing  <span class="keyword">import</span> MinMaxScaler</span><br><span class="line"><span class="comment"># initialization scaler</span></span><br><span class="line">scaler = MinMaxScaler()</span><br><span class="line">numberical = [<span class="string">'age'</span>,<span class="string">'education-num'</span>,<span class="string">'capital-gain'</span>,<span class="string">'capital-loss'</span>,<span class="string">'hours-per-week'</span>]</span><br><span class="line">features_raw[numberical] = scaler.fit_transform(data[numberical])</span><br><span class="line"><span class="comment"># category change to one hot encodeing</span></span><br><span class="line"><span class="keyword">from</span> sklearn <span class="keyword">import</span> preprocessing</span><br><span class="line">income = pd.Series(preprocessing.LabelEncoder().fit_transform(income_raw))</span><br><span class="line">features = pd.get_dummies(features_raw)</span><br></pre></td></tr></table></figure>
<h3 id="混洗和切分数据"><a href="#混洗和切分数据" class="headerlink" title="混洗和切分数据"></a>混洗和切分数据</h3><p>现在所有的 <em>类别变量</em> 已被转换成数值特征，而且所有的数值特征已被规一化。和我们一般情况下做的一样，我们现在将数据（包括特征和它们的标签）切分成训练和测试集。其中80%的数据将用于训练和20%的数据用于测试。然后再进一步把训练数据分为训练集和验证集，用来选择和优化模型。<br>用了stratify参数，training集和testing集的类的比例是 A：B= 4：1，等同于split前的比例（80：20）。通常在这种类分布不平衡的情况下会用到stratify。</p>
<p>将stratify=X就是按照X中的比例分配</p>
<p>将stratify=y就是按照y中的比例分配<br><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">from</span> sklearn <span class="keyword">import</span> train_test_split</span><br><span class="line">X_train,X_test,y_train,y_test = train_test_split(features,income,test_size=<span class="number">0.2</span>,random_state=<span class="number">0</span>,stratify=income)</span><br><span class="line">X_train,X_val,y_train,y_val = train_test_split(X_train,y_train,test_size=<span class="number">0.2</span>,random_state=<span class="number">0</span>,stratify=y_train)</span><br><span class="line"><span class="comment"># 显示切分的结果</span></span><br><span class="line"><span class="keyword">print</span> (<span class="string">"Training set has &#123;&#125; samples."</span>.format(X_train.shape[<span class="number">0</span>]))</span><br><span class="line"><span class="keyword">print</span> (<span class="string">"Validation set has &#123;&#125; samples."</span>.format(X_val.shape[<span class="number">0</span>]))</span><br><span class="line"><span class="keyword">print</span> (<span class="string">"Testing set has &#123;&#125; samples."</span>.format(X_test.shape[<span class="number">0</span>]))</span><br></pre></td></tr></table></figure></p>
<h2 id="评价模型性能"><a href="#评价模型性能" class="headerlink" title="评价模型性能"></a>评价模型性能</h2><p>在这一部分中，我们将尝试四种不同的算法，并确定哪一个能够最好地建模数据。四种算法包含一个天真的预测器 和三个你选择的监督学习器。</p>
<h3 id="评价方法和朴素的预测器"><a href="#评价方法和朴素的预测器" class="headerlink" title="评价方法和朴素的预测器"></a>评价方法和朴素的预测器</h3><p><em>CharityML</em>通过他们的研究人员知道被调查者的年收入大于\$50,000最有可能向他们捐款。因为这个原因<em>CharityML</em>对于准确预测谁能够获得\$50,000以上收入尤其有兴趣。这样看起来使用<strong>准确率</strong>作为评价模型的标准是合适的。另外，把<em>没有</em>收入大于\$50,000的人识别成年收入大于\$50,000对于<em>CharityML</em>来说是有害的，因为他想要找到的是有意愿捐款的用户。这样，我们期望的模型具有准确预测那些能够年收入大于\$50,000的能力比模型去<strong>查全</strong>这些被调查者<em>更重要</em>。我们能够使用<strong>F-beta score</strong>作为评价指标，这样能够同时考虑查准率和查全率：<br>$$F_{\beta}=(1+\beta^2)\cdot\frac{precision\cdot recall}{\beta^2 \cdot precision + recall}$$<br>尤其是，当 $\beta = 0.5$ 的时候更多的强调查准率，这叫做<strong>$F_{0.5}$ score</strong> （或者为了简单叫做F-score）。</p>
<h3 id="朴素预测器的性能"><a href="#朴素预测器的性能" class="headerlink" title="朴素预测器的性能"></a>朴素预测器的性能</h3><p>通过初步探索收入超过和不超过50，000美金的人数，可以看出超过50，000美金的人数并不多。如果预测所有人全部都超过50，000美金，那么准确率也会超过50%。这种预测器被称为朴素预测器，这样的一个预测器是评判一个模型表现的基线。</p>
<p>在不使用scikit-learn,需要自己根据公式计算。<br>true positive （TP：正类判定为正类）判定正确<br>false positive（FP：负类判定为正类）<br>false negative（FN：正类判定为负类）<br>True negative （TN：负类判定为负类）判定正确<br>错误率 error rate = （FP+FN）/（TN+TP+FN+FP）<br>准确率 accuracy = （TP+TN）/（TN+TP+FN+FP）<br>精准率 precision = TP/（TP+FP）<br>召回率 recall = TP/（TP+FN）<br><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">TP = float(len(y_val[y_val==<span class="number">1</span>]))</span><br><span class="line">FP = float(len(y_val[y_val==<span class="number">0</span>]))</span><br><span class="line">FN = <span class="number">0</span></span><br><span class="line">TN = <span class="number">0</span></span><br><span class="line">accuracy = (TP+TN)/(TP+FP+FN+TN)</span><br><span class="line">precision = TP/(TP+FP)</span><br><span class="line">recall = TP/(TP+FN)</span><br><span class="line">f_score = (<span class="number">1</span>+<span class="number">0.5</span>**<span class="number">2</span>)*precision*recall/((<span class="number">0.5</span>**<span class="number">2</span>)*precision+recall)</span><br></pre></td></tr></table></figure></p>
<p>当$\beta = 0.5$ 的时候更多的强调查准率，这叫做$F_{0.5 score}$或者简称为$F{-score}$。</p>
<h1 id="监督学习模型"><a href="#监督学习模型" class="headerlink" title="监督学习模型"></a>监督学习模型</h1><h3 id="模型1-支持向量机（SVM）"><a href="#模型1-支持向量机（SVM）" class="headerlink" title="模型1 支持向量机（SVM）"></a>模型1 支持向量机（SVM）</h3><p>选用kernel为了linear和rbf.现实中SVM广泛应用于视频和图像中人类行为的识别，根据图像判断人在干什么。<br>参考文献：<a href="http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=1334462" target="_blank" rel="noopener">http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=1334462</a><br>模型的优缺点：<br>优点：</p>
<ul>
<li>在非线性可分问题上表现优秀。</li>
<li>在训练数据量较小的表现良好。</li>
<li>抗数据的攻击能力好。<br>缺点：</li>
<li>在数据量较大的情况下，训练时间长导致难以实施。</li>
<li>解决多分类问题存在一定的困难。</li>
</ul>
<p>结合当前的finding donors 数据集的特点分析如下：</p>
<ul>
<li>首先，当前数据集的不是很大，属于小数据集。</li>
<li>其次，数据是否是线性的还未知。</li>
<li>最后预测目标是二分类问题，不是多分类，所以适合SVM<h3 id="K近邻"><a href="#K近邻" class="headerlink" title="K近邻"></a>K近邻</h3>使用K近邻法估计和绘制森林密度，体积和覆盖类型<br>参考文献：<a href="http://www.sciencedirect.com/science/article/pii/S0034425701002097" target="_blank" rel="noopener">http://www.sciencedirect.com/science/article/pii/S0034425701002097</a><br>模型的优缺点：<br>优点：</li>
<li>思想简单，容易理解和解释。</li>
<li>聚类效果不错。<br>缺点：</li>
<li>对异常值敏感。</li>
<li>需要提前确定k值。</li>
<li>局部最优解不是全局最优解（和初值的选取有关）。</li>
<li>算法复杂度不易控制O(NKm),迭代次数可能会很大，（考虑到m的值可能会很大）。<br>结合当前的finding donors 数据集的特点分析如下：</li>
<li>我们的数据可能存在异常值，但是可以剔除，降低对异常值的敏感程度。</li>
<li>数据量不大，所以计算时间不会太长。</li>
<li>因为是二分类，所以k值可以确定。<h3 id="集成方法（选择Random-Forest）"><a href="#集成方法（选择Random-Forest）" class="headerlink" title="集成方法（选择Random Forest）"></a>集成方法（选择Random Forest）</h3>应用场景：</li>
<li>随机森林应用于土地覆盖率</li>
<li>参考文献： <a href="http://www.sciencedirect.com/science/article/pii/S0924271611001304" target="_blank" rel="noopener">http://www.sciencedirect.com/science/article/pii/S0924271611001304</a><br>模型的优缺点：<br>优点：</li>
<li>准确率高。</li>
<li>能够有效的运行在大数据中。</li>
<li>能处理高维特征的输入样本，不需要降维。</li>
<li>能够评估各个特征在分类问题上的重要性。<br>缺点：</li>
<li>随机森林在一些噪音较大的分类和回归问题上会过拟合。</li>
<li>对于有不同取值的属性的数据，取值划分较多的属性会对随机森林产生更大的影响，所以随机森林在这种数据上产生的的属性权值是不可信的。<br>结合当前数据集，综合评价：</li>
<li>数据维度较高，处理起来很合适。</li>
<li>能够评估各个特征的重要性，可用作为一个很好的特征参考。</li>
<li>准确率较高，小范围的噪音不会过拟合。<h2 id="可视化脚步"><a href="#可视化脚步" class="headerlink" title="可视化脚步"></a>可视化脚步</h2><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br><span class="line">98</span><br><span class="line">99</span><br><span class="line">100</span><br><span class="line">101</span><br><span class="line">102</span><br><span class="line">103</span><br><span class="line">104</span><br><span class="line">105</span><br><span class="line">106</span><br><span class="line">107</span><br><span class="line">108</span><br><span class="line">109</span><br><span class="line">110</span><br><span class="line">111</span><br><span class="line">112</span><br><span class="line">113</span><br><span class="line">114</span><br><span class="line">115</span><br><span class="line">116</span><br><span class="line">117</span><br><span class="line">118</span><br><span class="line">119</span><br><span class="line">120</span><br><span class="line">121</span><br><span class="line">122</span><br><span class="line">123</span><br><span class="line">124</span><br><span class="line">125</span><br><span class="line">126</span><br><span class="line">127</span><br><span class="line">128</span><br><span class="line">129</span><br><span class="line">130</span><br><span class="line">131</span><br><span class="line">132</span><br><span class="line">133</span><br><span class="line">134</span><br><span class="line">135</span><br><span class="line">136</span><br><span class="line">137</span><br><span class="line">138</span><br><span class="line">139</span><br><span class="line">140</span><br><span class="line">141</span><br><span class="line">142</span><br><span class="line">143</span><br><span class="line">144</span><br><span class="line">145</span><br><span class="line">146</span><br><span class="line">147</span><br><span class="line">148</span><br><span class="line">149</span><br><span class="line">150</span><br><span class="line">151</span><br><span class="line">152</span><br><span class="line">153</span><br><span class="line">154</span><br><span class="line">155</span><br><span class="line">156</span><br><span class="line">157</span><br><span class="line">158</span><br><span class="line">159</span><br><span class="line">160</span><br><span class="line">161</span><br><span class="line">162</span><br><span class="line">163</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">###########################################</span></span><br><span class="line"><span class="comment"># Suppress matplotlib user warnings</span></span><br><span class="line"><span class="comment"># Necessary for newer version of matplotlib</span></span><br><span class="line"><span class="keyword">import</span> warnings</span><br><span class="line">warnings.filterwarnings(<span class="string">"ignore"</span>, category = UserWarning, module = <span class="string">"matplotlib"</span>)</span><br><span class="line"><span class="comment">#</span></span><br><span class="line"><span class="comment"># Display inline matplotlib plots with IPython</span></span><br><span class="line"><span class="keyword">from</span> IPython <span class="keyword">import</span> get_ipython</span><br><span class="line">get_ipython().run_line_magic(<span class="string">'matplotlib'</span>, <span class="string">'inline'</span>)</span><br><span class="line"><span class="comment">###########################################</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> matplotlib.pyplot <span class="keyword">as</span> pl</span><br><span class="line"><span class="keyword">import</span> matplotlib.patches <span class="keyword">as</span> mpatches</span><br><span class="line"><span class="keyword">import</span> numpy <span class="keyword">as</span> np</span><br><span class="line"><span class="keyword">import</span> pandas <span class="keyword">as</span> pd</span><br><span class="line"><span class="keyword">from</span> time <span class="keyword">import</span> time</span><br><span class="line"><span class="keyword">from</span> sklearn.metrics <span class="keyword">import</span> f1_score, accuracy_score</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">distribution</span><span class="params">(data, transformed = False)</span>:</span></span><br><span class="line">    <span class="string">"""</span></span><br><span class="line"><span class="string">    Visualization code for displaying skewed distributions of features</span></span><br><span class="line"><span class="string">    """</span></span><br><span class="line"></span><br><span class="line">    <span class="comment"># Create figure</span></span><br><span class="line">    fig = pl.figure(figsize = (<span class="number">11</span>,<span class="number">5</span>));</span><br><span class="line"></span><br><span class="line">    <span class="comment"># Skewed feature plotting</span></span><br><span class="line">    <span class="keyword">for</span> i, feature <span class="keyword">in</span> enumerate([<span class="string">'capital-gain'</span>,<span class="string">'capital-loss'</span>]):</span><br><span class="line">        ax = fig.add_subplot(<span class="number">1</span>, <span class="number">2</span>, i+<span class="number">1</span>)</span><br><span class="line">        ax.hist(data[feature], bins = <span class="number">25</span>, color = <span class="string">'#00A0A0'</span>)</span><br><span class="line">        ax.set_title(<span class="string">"'%s' Feature Distribution"</span>%(feature), fontsize = <span class="number">14</span>)</span><br><span class="line">        ax.set_xlabel(<span class="string">"Value"</span>)</span><br><span class="line">        ax.set_ylabel(<span class="string">"Number of Records"</span>)</span><br><span class="line">        ax.set_ylim((<span class="number">0</span>, <span class="number">2000</span>))</span><br><span class="line">        ax.set_yticks([<span class="number">0</span>, <span class="number">500</span>, <span class="number">1000</span>, <span class="number">1500</span>, <span class="number">2000</span>])</span><br><span class="line">        ax.set_yticklabels([<span class="number">0</span>, <span class="number">500</span>, <span class="number">1000</span>, <span class="number">1500</span>, <span class="string">"&gt;2000"</span>])</span><br><span class="line"></span><br><span class="line">    <span class="comment"># Plot aesthetics</span></span><br><span class="line">    <span class="keyword">if</span> transformed:</span><br><span class="line">        fig.suptitle(<span class="string">"Log-transformed Distributions of Continuous Census Data Features"</span>, \</span><br><span class="line">            fontsize = <span class="number">16</span>, y = <span class="number">1.03</span>)</span><br><span class="line">    <span class="keyword">else</span>:</span><br><span class="line">        fig.suptitle(<span class="string">"Skewed Distributions of Continuous Census Data Features"</span>, \</span><br><span class="line">            fontsize = <span class="number">16</span>, y = <span class="number">1.03</span>)</span><br><span class="line"></span><br><span class="line">    fig.tight_layout()</span><br><span class="line">    fig.show()</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">evaluate</span><span class="params">(results, accuracy, f1)</span>:</span></span><br><span class="line">    <span class="string">"""</span></span><br><span class="line"><span class="string">    Visualization code to display results of various learners.</span></span><br><span class="line"><span class="string"></span></span><br><span class="line"><span class="string">    inputs:</span></span><br><span class="line"><span class="string">      - learners: a list of supervised learners</span></span><br><span class="line"><span class="string">      - stats: a list of dictionaries of the statistic results from 'train_predict()'</span></span><br><span class="line"><span class="string">      - accuracy: The score for the naive predictor</span></span><br><span class="line"><span class="string">      - f1: The score for the naive predictor</span></span><br><span class="line"><span class="string">    """</span></span><br><span class="line"></span><br><span class="line">    <span class="comment"># Create figure</span></span><br><span class="line">    fig, ax = pl.subplots(<span class="number">2</span>, <span class="number">3</span>, figsize = (<span class="number">11</span>,<span class="number">7</span>))</span><br><span class="line"></span><br><span class="line">    <span class="comment"># Constants</span></span><br><span class="line">    bar_width = <span class="number">0.3</span></span><br><span class="line">    colors = [<span class="string">'#A00000'</span>,<span class="string">'#00A0A0'</span>,<span class="string">'#00A000'</span>]</span><br><span class="line"></span><br><span class="line">    <span class="comment"># Super loop to plot four panels of data</span></span><br><span class="line">    <span class="keyword">for</span> k, learner <span class="keyword">in</span> enumerate(results.keys()):</span><br><span class="line">        <span class="keyword">for</span> j, metric <span class="keyword">in</span> enumerate([<span class="string">'train_time'</span>, <span class="string">'acc_train'</span>, <span class="string">'f_train'</span>, <span class="string">'pred_time'</span>, <span class="string">'acc_val'</span>, <span class="string">'f_val'</span>]):</span><br><span class="line">            <span class="keyword">for</span> i <span class="keyword">in</span> np.arange(<span class="number">3</span>):</span><br><span class="line"></span><br><span class="line">                <span class="comment"># Creative plot code</span></span><br><span class="line">                ax[j/<span class="number">3</span>, j%<span class="number">3</span>].bar(i+k*bar_width, results[learner][i][metric], width = bar_width, color = colors[k])</span><br><span class="line">                ax[j/<span class="number">3</span>, j%<span class="number">3</span>].set_xticks([<span class="number">0.45</span>, <span class="number">1.45</span>, <span class="number">2.45</span>])</span><br><span class="line">                ax[j/<span class="number">3</span>, j%<span class="number">3</span>].set_xticklabels([<span class="string">"1%"</span>, <span class="string">"10%"</span>, <span class="string">"100%"</span>])</span><br><span class="line">                ax[j/<span class="number">3</span>, j%<span class="number">3</span>].set_xlabel(<span class="string">"Training Set Size"</span>)</span><br><span class="line">                ax[j/<span class="number">3</span>, j%<span class="number">3</span>].set_xlim((<span class="number">-0.1</span>, <span class="number">3.0</span>))</span><br><span class="line"></span><br><span class="line">    <span class="comment"># Add unique y-labels</span></span><br><span class="line">    ax[<span class="number">0</span>, <span class="number">0</span>].set_ylabel(<span class="string">"Time (in seconds)"</span>)</span><br><span class="line">    ax[<span class="number">0</span>, <span class="number">1</span>].set_ylabel(<span class="string">"Accuracy Score"</span>)</span><br><span class="line">    ax[<span class="number">0</span>, <span class="number">2</span>].set_ylabel(<span class="string">"F-score"</span>)</span><br><span class="line">    ax[<span class="number">1</span>, <span class="number">0</span>].set_ylabel(<span class="string">"Time (in seconds)"</span>)</span><br><span class="line">    ax[<span class="number">1</span>, <span class="number">1</span>].set_ylabel(<span class="string">"Accuracy Score"</span>)</span><br><span class="line">    ax[<span class="number">1</span>, <span class="number">2</span>].set_ylabel(<span class="string">"F-score"</span>)</span><br><span class="line"></span><br><span class="line">    <span class="comment"># Add titles</span></span><br><span class="line">    ax[<span class="number">0</span>, <span class="number">0</span>].set_title(<span class="string">"Model Training"</span>)</span><br><span class="line">    ax[<span class="number">0</span>, <span class="number">1</span>].set_title(<span class="string">"Accuracy Score on Training Subset"</span>)</span><br><span class="line">    ax[<span class="number">0</span>, <span class="number">2</span>].set_title(<span class="string">"F-score on Training Subset"</span>)</span><br><span class="line">    ax[<span class="number">1</span>, <span class="number">0</span>].set_title(<span class="string">"Model Predicting"</span>)</span><br><span class="line">    ax[<span class="number">1</span>, <span class="number">1</span>].set_title(<span class="string">"Accuracy Score on Validation Set"</span>)</span><br><span class="line">    ax[<span class="number">1</span>, <span class="number">2</span>].set_title(<span class="string">"F-score on Validation Set"</span>)</span><br><span class="line"></span><br><span class="line">    <span class="comment"># Add horizontal lines for naive predictors</span></span><br><span class="line">    ax[<span class="number">0</span>, <span class="number">1</span>].axhline(y = accuracy, xmin = <span class="number">-0.1</span>, xmax = <span class="number">3.0</span>, linewidth = <span class="number">1</span>, color = <span class="string">'k'</span>, linestyle = <span class="string">'dashed'</span>)</span><br><span class="line">    ax[<span class="number">1</span>, <span class="number">1</span>].axhline(y = accuracy, xmin = <span class="number">-0.1</span>, xmax = <span class="number">3.0</span>, linewidth = <span class="number">1</span>, color = <span class="string">'k'</span>, linestyle = <span class="string">'dashed'</span>)</span><br><span class="line">    ax[<span class="number">0</span>, <span class="number">2</span>].axhline(y = f1, xmin = <span class="number">-0.1</span>, xmax = <span class="number">3.0</span>, linewidth = <span class="number">1</span>, color = <span class="string">'k'</span>, linestyle = <span class="string">'dashed'</span>)</span><br><span class="line">    ax[<span class="number">1</span>, <span class="number">2</span>].axhline(y = f1, xmin = <span class="number">-0.1</span>, xmax = <span class="number">3.0</span>, linewidth = <span class="number">1</span>, color = <span class="string">'k'</span>, linestyle = <span class="string">'dashed'</span>)</span><br><span class="line"></span><br><span class="line">    <span class="comment"># Set y-limits for score panels</span></span><br><span class="line">    ax[<span class="number">0</span>, <span class="number">1</span>].set_ylim((<span class="number">0</span>, <span class="number">1</span>))</span><br><span class="line">    ax[<span class="number">0</span>, <span class="number">2</span>].set_ylim((<span class="number">0</span>, <span class="number">1</span>))</span><br><span class="line">    ax[<span class="number">1</span>, <span class="number">1</span>].set_ylim((<span class="number">0</span>, <span class="number">1</span>))</span><br><span class="line">    ax[<span class="number">1</span>, <span class="number">2</span>].set_ylim((<span class="number">0</span>, <span class="number">1</span>))</span><br><span class="line"></span><br><span class="line">    <span class="comment"># Create patches for the legend</span></span><br><span class="line">    patches = []</span><br><span class="line">    <span class="keyword">for</span> i, learner <span class="keyword">in</span> enumerate(results.keys()):</span><br><span class="line">        patches.append(mpatches.Patch(color = colors[i], label = learner))</span><br><span class="line">    pl.legend(handles = patches, bbox_to_anchor = (<span class="number">-.80</span>, <span class="number">2.53</span>), \</span><br><span class="line">               loc = <span class="string">'upper center'</span>, borderaxespad = <span class="number">0.</span>, ncol = <span class="number">3</span>, fontsize = <span class="string">'x-large'</span>)</span><br><span class="line"></span><br><span class="line">    <span class="comment"># Aesthetics</span></span><br><span class="line">    pl.suptitle(<span class="string">"Performance Metrics for Three Supervised Learning Models"</span>, fontsize = <span class="number">16</span>, y = <span class="number">1.10</span>)</span><br><span class="line">    pl.tight_layout()</span><br><span class="line">    pl.show()</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">feature_plot</span><span class="params">(importances, X_train, y_train)</span>:</span></span><br><span class="line"></span><br><span class="line">    <span class="comment"># Display the five most important features</span></span><br><span class="line">    indices = np.argsort(importances)[::<span class="number">-1</span>]</span><br><span class="line">    columns = X_train.columns.values[indices[:<span class="number">5</span>]]</span><br><span class="line">    values = importances[indices][:<span class="number">5</span>]</span><br><span class="line"></span><br><span class="line">    <span class="comment"># Creat the plot</span></span><br><span class="line">    fig = pl.figure(figsize = (<span class="number">9</span>,<span class="number">5</span>))</span><br><span class="line">    pl.title(<span class="string">"Normalized Weights for First Five Most Predictive Features"</span>, fontsize = <span class="number">16</span>)</span><br><span class="line">    rects = pl.bar(np.arange(<span class="number">5</span>), values, width = <span class="number">0.6</span>, align=<span class="string">"center"</span>, color = <span class="string">'#00A000'</span>, \</span><br><span class="line">                label = <span class="string">"Feature Weight"</span>)</span><br><span class="line"></span><br><span class="line">    <span class="comment"># make bar chart higher to fit the text label</span></span><br><span class="line">    axes = pl.gca()</span><br><span class="line">    axes.set_ylim([<span class="number">0</span>, np.max(values) * <span class="number">1.1</span>])</span><br><span class="line"></span><br><span class="line">    <span class="comment"># add text label on each bar</span></span><br><span class="line">    delta = np.max(values) * <span class="number">0.02</span></span><br><span class="line"></span><br><span class="line">    <span class="keyword">for</span> rect <span class="keyword">in</span> rects:</span><br><span class="line">        height = rect.get_height()</span><br><span class="line">        pl.text(rect.get_x() + rect.get_width()/<span class="number">2.</span>,</span><br><span class="line">                height + delta,</span><br><span class="line">                <span class="string">'%.2f'</span> % height,</span><br><span class="line">                ha=<span class="string">'center'</span>,</span><br><span class="line">                va=<span class="string">'bottom'</span>)</span><br><span class="line"></span><br><span class="line">    <span class="comment"># Detect if xlabels are too long</span></span><br><span class="line">    rotation = <span class="number">0</span></span><br><span class="line">    <span class="keyword">for</span> i <span class="keyword">in</span> columns:</span><br><span class="line">        <span class="keyword">if</span> len(i) &gt; <span class="number">20</span>:</span><br><span class="line">            rotation = <span class="number">10</span> <span class="comment"># If one is longer than 20 than rotate 10 degrees</span></span><br><span class="line">            <span class="keyword">break</span></span><br><span class="line">    pl.xticks(np.arange(<span class="number">5</span>), columns, rotation = rotation)</span><br><span class="line">    pl.xlim((<span class="number">-0.5</span>, <span class="number">4.5</span>))</span><br><span class="line">    pl.ylabel(<span class="string">"Weight"</span>, fontsize = <span class="number">12</span>)</span><br><span class="line">    pl.xlabel(<span class="string">"Feature"</span>, fontsize = <span class="number">12</span>)</span><br><span class="line"></span><br><span class="line">    pl.legend(loc = <span class="string">'upper center'</span>)</span><br><span class="line">    pl.tight_layout()</span><br><span class="line">    pl.show()</span><br></pre></td></tr></table></figure>
</li>
</ul>

      
    </div>
    <footer class="article-footer">
      <a data-url="http://yoursite.com/2018/07/10/finding-donors-2nd/" data-id="cjptgn1m40009ev15wd66zze6" class="article-share-link">Share</a>
      
      
    </footer>
  </div>
  
    
<nav id="article-nav">
  
    <a href="/2018/07/10/finding-donors-1st/" id="article-nav-newer" class="article-nav-link-wrap">
      <strong class="article-nav-caption">Newer</strong>
      <div class="article-nav-title">
        
          finding_donors_1st
        
      </div>
    </a>
  
  
    <a href="/2018/07/10/change-themes/" id="article-nav-older" class="article-nav-link-wrap">
      <strong class="article-nav-caption">Older</strong>
      <div class="article-nav-title">change_themes</div>
    </a>
  
</nav>

  
</article>

</section>
        
          <aside id="sidebar">
  
    

  
    
  <div class="widget-wrap">
    <h3 class="widget-title">Tags</h3>
    <div class="widget">
      <ul class="tag-list"><li class="tag-list-item"><a class="tag-list-link" href="/tags/Machine-Learning/">Machine Learning</a></li><li class="tag-list-item"><a class="tag-list-link" href="/tags/linear-regression/">linear regression</a></li></ul>
    </div>
  </div>


  
    
  <div class="widget-wrap">
    <h3 class="widget-title">Tag Cloud</h3>
    <div class="widget tagcloud">
      <a href="/tags/Machine-Learning/" style="font-size: 20px;">Machine Learning</a> <a href="/tags/linear-regression/" style="font-size: 10px;">linear regression</a>
    </div>
  </div>

  
    
  <div class="widget-wrap">
    <h3 class="widget-title">Archives</h3>
    <div class="widget">
      <ul class="archive-list"><li class="archive-list-item"><a class="archive-list-link" href="/archives/2018/12/">December 2018</a></li><li class="archive-list-item"><a class="archive-list-link" href="/archives/2018/08/">August 2018</a></li><li class="archive-list-item"><a class="archive-list-link" href="/archives/2018/07/">July 2018</a></li></ul>
    </div>
  </div>


  
    
  <div class="widget-wrap">
    <h3 class="widget-title">Recent Posts</h3>
    <div class="widget">
      <ul>
        
          <li>
            <a href="/2018/12/18/SGD/">SGD</a>
          </li>
        
          <li>
            <a href="/2018/12/18/gradient-descent/">gradient_descent</a>
          </li>
        
          <li>
            <a href="/2018/12/06/linear-regression-gluon/">linear-regression-gluon</a>
          </li>
        
          <li>
            <a href="/2018/12/06/linear-regression-sources-md/">linear-regression-sources.md</a>
          </li>
        
          <li>
            <a href="/2018/12/05/regression/">regression</a>
          </li>
        
      </ul>
    </div>
  </div>

  
</aside>
        
      </div>
      <footer id="footer">
  
  <div class="outer">
    <div id="footer-info" class="inner">
      &copy; 2018 alex Chen<br>
      Powered by <a href="http://hexo.io/" target="_blank">Hexo</a>
    </div>
  </div>
</footer>
    </div>
    <nav id="mobile-nav">
  
    <a href="/" class="mobile-nav-link">Home</a>
  
    <a href="/archives" class="mobile-nav-link">Archives</a>
  
</nav>
    

<script src="//ajax.googleapis.com/ajax/libs/jquery/2.0.3/jquery.min.js"></script>


  <link rel="stylesheet" href="/fancybox/jquery.fancybox.css">
  <script src="/fancybox/jquery.fancybox.pack.js"></script>


<script src="/js/script.js"></script>



  </div>
</body>
</html>